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DETAILED ACTION 
Double Patenting 



The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g.. In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 1 1 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vagal, 422 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 



Claims 1-36 are rejected under the judicially created doctrine of obviousness-type 
double patenting as being unpatentable over claims 1-42 of U.S. Patent No. 6,853,398. 
An obviousness-type double patenting rejection is appropriate where the conflicting 
claims are not identical, but an examined application claim is not patentably distinct 
from the reference claim(s) because the examined claim is either anticipated by, or 
would have been obvious over, the reference claim(s). See, e.g., In re Berg. Although 
the conflicting claims are not identical, they are not patentably distinct from each other 
because claims 1-36 falls within the scope of the method and system for real-time video 
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communication within a virtual environment recited in claims 1-42 of U.S. Patent No. 
6,853,398. That is, claims 1-32 is obvious over claims 1-42 of U.S. Patent No. 
6,853,398. Specifically, since a gaming environment are a species of the generic 
category defined by "virtual environment," the system and method of claims 1-36 
reciting a "gaming environment" is obvious over claims 1-42 of U.S. Patent No. 
6,853,398 reciting a "virtual environment." 

Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the Invention was described in (1) an application for patent, published under section 122(b), by 
another filed In the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed In the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1-19 and 21-36 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Malzbender et al. (US 6,853,398 B2), "Malzbender". 

The applied reference has common inventors with the instant application. Based 
upon the earlier effective U.S. filing date of the reference, it constitutes prior art under 
35 U.S.C. 102(e). This rejection under 35 U.S.C. 102(e) might be overcome either by a 
showing under 37 CFR 1 .132 that any invention disclosed but not claimed in the 
reference was derived from the inventor of this application and is thus not the invention 
"by another," or by an appropriate showing under 37 CFR 1.131. 
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Claims 1. 21: Malzbender discloses 

• capturing a plurality of real-time video streams of a local participant from a 
plurality of camera viewpoints (Malzbender, Abstract); 

• generating real-time renderings of said local participant based on said plurality of 
real-time video streams by applying a new view synthesis technique, said real- 
time renderings taken from a perspective of a remote participant located remotely 
in said gaming environment (Malzbender, Abstract); and 

• sending said real-time renderings to said remote participant for viewing within 
said gaming environment (Malzbender, Abstract). 

Claim 2: Malzbender discloses incorporating said real-time renderings 
comprising real-time photo-realistic features of said local participant into a graphically 
generated avatar (col. 4, lines 35-46), said graphically generated avatar representing 
said local participant in said gaming environment; and sending said graphically 
generated avatar incorporating said real-time renderings to said local participant for 
viewing within said gaming environment (col. 4, lines 35-46). 

Claim 3: Malzbender discloses generating real-time renderings of said local 
participant further comprises: generating image based renderings of said local 
participant as said real-time renderings, wherein said new-view synthesis technique 
comprises an image-based visual hull technique. The image-based visual hull 
technique, as disclosed by the Applicant, "back projects the contour silhouettes into a 
three-dimensional space and computes the intersection of the resulting frusta. The 
intersection, the visual hull, approximates the geometry of the user. Rendering this 
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geometry with view-dependent texture mapping creates convincing new views" 
(Applicant, paragraph 50). Malzbender equally discloses, "multiple images are collected 
without a foreground participant present to build a background model for each pixel in 
each camera. This data may be stored in module 130 for purposes of contour 
extraction. The model is a statistical characterization that specifies what the mean and 
standard deviations for each color channel are at each sensor element when viewing 
the background. This allows for the segmentation of the user from the background in 
each of the camera views. Other background models can easily be implemented, such 
as, multimodal representations of color statistics, and mixtures of Gaussian distribution 
functions" (col. 6, lines 30-41). 

Claim 4: Malzbender discloses generating real-time renderings of said local 
participant further comprises: generating a three-dimensional geometric model of said 
local participant from said plurality of real-time video streams as said real-time 
renderings (col. 7, lines 8-15). 

Claim 5: Malzbender discloses generating real-time renderings of said local 
participant further comprises: rendering said real-time renderings from a second 
location of said remote participant with respect to a first location of said local participant 
within a coordinate space of said gaming environment (col. 5, lines 45-63). 

Claim 6: Malzbender discloses enabling a change in at least one of said first and . 
second locations by allowing said local participant and said remote participant to 
navigate through said coordinate space; and modifying said real-time renderings to 
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reflect said change in at least one of said first and second locations (col. 5, line 45 - col. 
6, line 2). 

Claims 7, 22: Malzbender discloses blending said real-time renderings within a 
synthetic rendering of said gaming environment; and displaying said real-time 
renderings that is blended within said gaming environment at a display viewable by said 
second participant to enable real-time video communication for interactive gaming 
between said local participant and said remote participant within said gaming 
environment (col. 5, line 45 - col. 6, line 2). 

Claim 8: Malzbender discloses gaming environment comprises a three 
dimensional gaming environment (col. 7, lines 8-15). 

Claim 9: Malzbender discloses personifying said graphically generated avatar to 
said local participant by illustrating real-time emotion of said local participant through 
said video image stream comprising representative said real-time photo-realistic 
features of said local participant (col. 10, lines 47-58). 

Claim 10: Malzbender discloses generating another real-time renderings of said 
local participant based on said plurality of real-time video streams by applying said new 
view synthesis technique, said another real-time renderings taken from a perspective of 
another participant located remotely in said gaming environment; and sending said 
another real-time renderings to said another participant for viewing within said gaming 
environment (col. 9, lines 7-27). 

Claim 1 1 : Malzbender discloses 
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• capturing a plurality of real-time video streams of a local participant from a 
plurality of camera viewpoints (Malzbender, Abstract); 

• generating real-time renderings of said local participant based on said plurality of 
real-time video streams by applying a new view synthesis technique, said real- 
time renderings taken from a perspective of a remote participant located remotely 
in said gaming environment (Malzbender, Abstract); and 

• incorporating said video image stream comprising real-time photo-realistic 
features of said local participant into a graphically generated avatar (coL 4, lines 
35-46), said graphically generated avatar representing said local participant in 
said gaming environment (col. 4. lines 35-46). 

Claim 12: Malzbender discloses generating said video image stream further 
comprises: rendering said video image stream from a second location of said remote 
participant with respect to a first location of said local participant within a coordinate 
space of said gaming environment (col. 5, lines 45-63 and col. 9, lines 7-27). 

Claim 13: Malzbender discloses enabling a change in at least one of said first 
and second locations by allowing said local participant and said remote participant to 
navigate through said coordinate space; and modifying said video image stream to 
reflect said change in at least one of said first and second locations (col. 5, line 45 - col. 
6, line 2). 

Claim 14: Malzbender discloses blending said graphically generated avatar 
incorporating said video image stream within a synthetic rendering of said gaming 
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environment; and displaying said graphically generated avatar incorporating said video 
image stream within said gaming environment at a display viewable by said remote 
participant to enable real-time video communication for interactive gaming between said 
local participant and said remote participant within said gaming environment (col. 5, line 
45 - col. 6, line 2). 

Claim 15: Malzbender discloses personifying said graphically generated avatar to 
said local participant by illustrating real-time emotion of said local participant through 
said video image stream comprising representative said real-time photo-realistic 
features of said local participant (col. 10, lines 47-58). 

Claim 16: Malzbender discloses new-view synthesis technique comprises an 
image-based visual hull technique. The image-based visual hull technique, as disclosed 
by the Applicant, "back projects the contour silhouettes into a three-dimensional space 
and computes the intersection of the resulting frusta. The intersection, the visual hull, 
approximates the geometry of the user. Rendering this geometry with view-dependent 
texture mapping creates convincing new views" (Applicant, paragraph 50). Malzbender 
equally discloses, "multiple images are collected without a foreground participant 
present to build a background model for each pixel in each camera. This data may be 
stored in module 130 for purposes of contour extraction. The model is a statistical 
characterization that specifies what the mean and standard deviations for each color 
channel are at each sensor element when viewing the background. This allows for the 
segmentation of the user from the background in each of the camera views. Other 
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background models can easily be implemented, such as, multimodal representations of 
color statistics, and mixtures of Gaussian distribution functions" (col. 6, lines 30-41). 

Claim 17: Malzbender discloses generating said video image stream further 
comprises: generating a three dimensional geometric model of said local participant 
based on said plurality of real time video streams by applying said new view synthesis 
technique; and generating said video image stream of said local participant from said 
three dimensional geometric model (col. 7, lines 8-15). 

Claim 18: Malzbender discloses generating a real-time audio output from audio 
of said local participant to enable real-time audio communication in said gaming 
environment between said local participant and said remote participant (col. 8, lines 4- 
23). 

Claim 19: Malzbender discloses varying a total of said plurality of camera 
viewpoints when capturing said plurality of real-time video streams of said local 
participant to vary image quality in said plurality of video image streams (col. 5, lines 1- 
13). 

Claim 23: Malzbender discloses receiving a plurality of input audio streams in 
real-time associated with said plurality of observing participants for mixing into a single 
audio stream that is broadcast through a local speaker (Malzbender, claim 25). 

Claim 24: Malzbender discloses a display for viewing said plurality of graphically 
generated avatars representing said plurality of observing participants to enable real- 
time view communication between said local participant and said plurality of observing 
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participants within said three dimensional gaming environment {col. 6, line 58 - col. 7, 
line 7). 

Claim 25: Malzbender discloses a contour extraction module for extracting said 
local participant from a physical background (coL 5, lines 45-63). 

Claim 26: Malzbender discloses a means for capturing an audio stream in real- 
time of said local participant (col. 7, lines 39-45). 

Claim 27: Malzbender discloses a processor; and a computer readable memory 
coupled to said processor and containing program instructions that, when executed, 
implement a method of real-time rendering in a gaming environment to create an 
interactive experience, comprising: capturing a plurality of real-time video streams of a 
local participant from a plurality of camera viewpoints; generating real-time renderings 
of said local participant based on said plurality of real-time video streams by applying a 
new view synthesis technique, said real-time renderings taken from a perspective of a 
remote participant located remotely in said gaming environment; and sending said real- 
time renderings to said remote participant for viewing within said gaming environment 
(Malzbender, claim 33). 

Claim 28: Malzbender discloses incorporating said real-time renderings 
comprising real-time photo-realistic features of said local participant into a graphically 
generated avatar, said graphically generated avatar representing said local participant 
in said gaming environment; and sending said graphically generated avatar 
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incorporating said real-time renderings to said local participant for viewing within said 
gaming environment (col. 4, lines 35-46). 

Claim 29: Malzbender discloses generating real-time renderings of said local 
participant in said method further comprises: generating image based renderings of said 
local participant as said real-time renderings, wherein said new-view synthesis 
technique comprises an image-based visual hull technique. The image-based visual hull 
technique, as disclosed by the Applicant, "back projects the contour silhouettes into a 
three-dimensional space and computes the intersection of the resulting frusta. The 
intersection, the visual hull, approximates the geometry of the user. Rendering this 
geometry with view-dependent texture mapping creates convincing new views" 
(Applicant, paragraph 50). Malzbender equally discloses, "multiple images are collected 
without a foreground participant present to build a background model for each pixel in 
each camera. This data may be stored in module 130 for purposes of contour 
extraction. The model is a statistical characterization that specifies what the mean and 
standard deviations for each color channel are at each sensor element when viewing 
the background. This allows for the segmentation of the user from the background in 
each of the camera views. Other background models can easily be implemented, such 
as, multimodal representations of color statistics, and mixtures of Gaussian distribution 
functions" (col. 6, lines 30-41). 

Claim 30: Malzbender discloses generating real-time renderings of said local 
participant in said method further comprises: generating a three-dimensional geometric 
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model of said local participant from said plurality of real-time video streams as said real- 
time renderings (col. 7, lines 8-15). 

Claim 31 : Malzbender discloses generating real-time renderings of said local 
participant in said method further comprises: rendering said real-time renderings from a 
second location of said remote participant with respect to a first location of said local 
participant within a coordinate space of said gaming environment (col. 5, lines 45-63). 

Claim 32: Malzbender discloses enabling a change in at least one of said first 
and second locations by allowing said local participant and said remote participant to 
navigate through said coordinate space; and modifying said real-time renderings to 
reflect said change in at least one of said first and second locations (col. 5, line 45 - col. 
6, line 2). 

Claim 33: Malzbender discloses blending said real-time renderings within a 
synthetic rendering of said gaming environment; and displaying said real-time 
renderings that is blended within said gaming environment at a display viewable by said 
second participant to enable real-time video communication for interactive gaming 
between said local participant and said remote participant within said gaming 
environment (col. 5, line 45 - col. 6, line 2). 

Claim 34: Malzbender discloses wherein said gaming environment comprises a 
three dimensional gaming environment (col. 7, lines 8-15). 

Claim 35: Malzbender discloses personifying said graphically generated avatar to 
said local participant by illustrating real-time emotion of said local participant through 
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said video image stream comprising representative said real-time photo-realistic 
features of said local participant (col. 10, lines 47-58). 

Claim 36: Malzbender discloses generating another real-time renderings of said 
local participant based on said plurality of real-time video streams by applying said new 
view synthesis technique, said another real-time renderings taken from a perspective of 
another participant located remotely in said gaming environment; and sending said 
another real-time renderings to said another participant for viewing within said gaming 
environment (col. 9, lines 7-27). 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co,, 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 
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Claim 20 rejected under 35 U.S.C. 103(a) as being obvious over Malzbender et 
al. (US 6,853,398 B2), "Malzbender"'. 

The applied reference has a common assignee with the instant application. 
Based upon the earlier effective U.S. filing date of the reference, it constitutes prior art 
only under 35 U.S.C. 102(e). This rejection under 35 U.S.C. 103(a) might be overcome 
by: (1) a showing under 37 CFR 1.132 that any invention disclosed but not claimed in 
the reference was derived from the inventor of this application and is thus not an 
invention "by another"; (2) a showing of a date of invention for the claimed subject 
matter of the application which corresponds to subject matter disclosed but not claimed 
in the reference, prior to the effective U.S. filing date of the reference under 37 CFR 
1.131; or (3) an oath or declaration under 37 CFR 1.130 stating that the application and 
reference are currently owned by the same party and that the inventor named in the 
application is the prior inventor under 35 U.S.C. 104, together with a terminal disclaimer 
in accordance with 37 CFR 1 .321 (c). This rejection might also be overcome by showing 
that the reference is disqualified under 35 U.S.C. 103(c) as prior art in a rejection under 
35 U.S.C. 103(a). See MPEP § 706.02(l)(1) and § 706.02(l)(2). 

Claim 20: Malzbender does not disclose performing non-photorealistic 
processing of said plurality of video image streams resulting in a non-photorealistic 
cartoon rendering of said plurality of video image streams. Instead, Malzbender 
discloses performing photorealistic processing to render an avatar (coL 4, lines 35-46). 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to incorporate a cartoon rendering feature when Malzbender's invention is 
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applied to a gaming environment. It is old and well known in the art that gaming avatars 
notoriously non-photorealistic and often a depiction of something fantastical and/or 
inhuman (e.g. wizards, trolls, worms) to offer players with a non-real world experience. 
Therefore, it would have been obvious to one or ordinary skill in the art at the time of the 
instant invention to incorporate non-photorealistic processing to Malzbender's invention 
to provide players with a non-real world experience which attracts more players to play 
the game which increases game profits. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Kim whose telephone number is 571-272-1691, 
The examiner can normally be reached on M-F, 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Hotaling can be reached on 571-272-4437. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Infonnation regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct,uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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